Two Photon (Bruker) tutorial with holographic stimulation¶
This tutorial demonstraces how to access the Two Photon dataset using pynwb.
This dataset contains the Two Photon imaging, holographic stimulation data and behavior measurements from the ViRMEN system.
Contents:
- Reading an NWB file
- Access subject and task metadata
- Access Imaging
- Access Behavior
- Access Stimulus
- View NWB files
A schematic representation where the source data is saved in NWB:

Reading an NWB file ¶
This section demonstrates how to read an NWB file using pynwb.
Based on the NWB File Basics tutorial from PyNWB.
An NWBFile represents a single session of an experiment. Each NWBFile must have a session description, identifier, and session start time.
Reading is carried out using the NWBHDF5IO class. To read the NWB file use the read mode ("r") to retrieve an NWBFile object.
from pynwb import NWBHDF5IO
# The file path to a .nwb file
nwbfile_path = "/Volumes/t7-ssd/Pinto/nwbfiles/NCCR32_2023_02_20_Into_the_void_t_series_stim-000.nwb"
io = NWBHDF5IO(path=nwbfile_path, mode="r", load_namespaces=True)
nwbfile = io.read()
nwbfile
root (NWBFile)
file_create_date
acquisition (1)
TwoPhotonSeries
data
dimension
field_of_view
imaging_plane
optical_channel
device
stimulus (1)
HolographicStimulationSeries
data
timestamps
rois
table
id
imaging_plane
optical_channel
device
site
device
stimulus_pattern
device
spatial_light_modulator
light_source
processing (2)
behavior
data_interfaces (8)
Position
spatial_series (2)
SpatialSeries
data
timestamps
timestamp_link
SpatialSeriesByImFrame
data
timestamps
timestamp_link
SensorDots
data
timestamps
data
timestamps
timestamp_link
SensorDotsByImFrame
data
timestamps
data
timestamps
timestamp_link
Velocity
data
timestamps
data
timestamps
timestamp_link
VelocityByImFrame
data
timestamps
data
timestamps
timestamp_link
VelocityGain
data
timestamps
data
timestamps
timestamp_link
ViewAngle
spatial_series (3)
PositionViewAngleByImFrame
data
timestamps
data
timestamps
timestamp_link
VelocityViewAngle
data
timestamps
data
timestamps
timestamp_link
VelocityViewAngleByImFrame
data
timestamps
data
timestamps
timestamp_link
TTLs
id
ophys
data_interfaces (3)
Fluorescence
roi_response_series (3)
Deconvolved
data
rois
table
id
imaging_plane
optical_channel
device
Neuropil
data
rois
table
id
imaging_plane
optical_channel
device
RoiResponseSeries
data
rois
table
id
imaging_plane
optical_channel
device
ImageSegmentation
plane_segmentations (2)
PlaneSegmentation
id
imaging_plane
optical_channel
device
PlaneSegmentationHolographicStimulation
id
imaging_plane
optical_channel
device
SegmentationImages
images (2)
correlation
mean
devices (4)
BrukerFluorescenceMicroscope
device
light_source
spatial_light_modulator
imaging_planes (2)
ImagingPlane
optical_channel
device
ImagingPlaneHolographicStimulation
optical_channel
device
ogen_sites (1)
site
device
subject
lab_meta_data (2)
LabMetaData
mazes
id
stimulus_protocol
id
stimulus_pattern
Importantly, the session start time is the reference time for all timestamps in the file. For instance, an event with a timestamp of 0 in the file means the event occurred exactly at the session start time.
The session_start_time is extracted from the date and time variables from the ViRMEN file.
nwbfile.session_start_time
datetime.datetime(2023, 2, 20, 15, 58, 35, tzinfo=tzoffset(None, -28800))
nwbfile.subject
subject (SubjectExtension)
Access ViRMEN experimental metadata (mazes table, stimulus protocol parameters)¶
This section demonstrates how to access the task related metadata in an NWB file.
The ViRMEN experimental metadata is stored in a LabMetaData extension (ndx-pinto-metadata).
The LabMetaData](https://pynwb.readthedocs.io/en/stable/pynwb.file.html#pynwb.file.LabMetaData) object can be accessed as nwbfile.lab_meta_data["LabMetaData"].
The parameters for the mazes is added to the mazes table within nwbfile.lab_meta_data["LabMetaData"] which can be accessed as nwbfile.lab_meta_data["LabMetaData"].mazes.
Data arrays are read passively from the file. Accessing the data attribute of the mazes object does not read the data values, but presents an HDF5 object that can be indexed to read data. You can use the [:] operator to read the entire data array into memory as nwbfile.lab_meta_data["LabMetaData"].mazes[:].
nwbfile.lab_meta_data["LabMetaData"]
LabMetaData (LabMetaDataExtension)
mazes
id
stimulus_protocol
id
nwbfile.lab_meta_data["LabMetaData"].mazes[:]
| rewardFraction | antiFraction | world | maxTrialDuration | numTrials | numTrialsPerMin | criteriaNTrials | warmupNTrials | numSessions | performance | maxBias | warmupMaze | warmupPerform | warmupBias | warmupMotor | easyBlock | easyBlockNTrials | numBlockTrials | blockPerform | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| id | |||||||||||||||||||
| 0 | 1 | 0 | 1 | 1500 | 10 | 2 | inf | [] | 0 | 0 | inf | [] | [] | [] | [] | NaN | 10 | 20 | 0.7 |
nwbfile.lab_meta_data["LabMetaData"].stimulus_protocol[:]
| numMazesInProtocol | trialDraw | stimDraw | trialEndPauseDuration | interTrialCorrectDuration | interTrialWrongDuration | targetNumTrials | panSessionTrials | |
|---|---|---|---|---|---|---|---|---|
| id | ||||||||
| 0 | 1 | Random | Random | 1 | 1 | 3 | 5 | 0 |
Access trials¶
Behavior trials are stored in nwbfile.trials. The start_time denotes the start time of each trial in seconds relative to the global session start time (using the "StartOfTrial" column from ViRMEN .mat file).
The stop_time denotes the end time of each trial in seconds relative to the global session start time
(using the "EndOfTrial" column from the ViRMEN .mat file).
nwbfile.trials can be converted to a pandas DataFrame for convenient analysis using nwbfile.trials.to_dataframe.
trials = nwbfile.trials.to_dataframe()
trials[:10]
| start_time | stop_time | InterTrial | SetupTrial | InitializeTrial | WithinTrial | WithinContext | WithinCue | WithinDelay | WithinArms | ... | nSalient | nDistract | posSalient | startIdx | mazeID | mainMazeID | distanceTraveled | excessTravel | rewardFactor | reward | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| id | |||||||||||||||||||||
| 0 | 0.972257 | 9.469965 | 11.092663 | 0.000000 | 0.000000 | 7.842325 | 1.316735 | 2.925307 | 6.958919 | 7.850775 | ... | 1 | 0 | [14.691288945029548] | 1.0 | 1.0 | 1.0 | 91.593116 | -0.040910 | 2.0 | 1.0 |
| 1 | 13.482450 | 23.810661 | 24.893987 | 11.115410 | 13.451317 | 21.935497 | 13.801357 | 14.560017 | 20.968785 | 21.944084 | ... | 2 | 0 | [44.09698487410711, 38.86791360641561] | 1366.0 | 1.0 | 1.0 | 94.579079 | -0.009643 | 2.0 | 0.0 |
| 2 | 35.008142 | 41.995798 | 43.054207 | 24.912208 | 34.978534 | 39.545645 | 35.295272 | 36.178588 | 38.787250 | 39.553976 | ... | 1 | 0 | [42.11787931412284] | 3932.0 | 1.0 | 1.0 | 90.914928 | -0.048011 | 2.0 | 0.0 |
| 3 | 53.151482 | 68.236366 | 69.848842 | 43.074311 | 53.130345 | 57.364091 | 53.513897 | 54.022254 | 56.688921 | 57.372551 | ... | 1 | 0 | [31.783853729187047] | 6075.0 | 1.0 | 1.0 | 92.609304 | -0.030269 | 2.0 | 1.0 |
| 4 | 71.959344 | 81.675022 | 82.749953 | 69.873477 | 71.932397 | 75.490970 | 72.199315 | 72.724034 | 74.924241 | 75.499314 | ... | 2 | 0 | [14.271608632711335, 33.884239554294254] | 8242.0 | 1.0 | 1.0 | 96.921628 | 0.014886 | 2.0 | 0.0 |
| 5 | 92.866958 | 101.618587 | 102.676910 | 82.788998 | 92.851003 | 100.118571 | 95.401442 | 96.151644 | 99.143451 | 100.126880 | ... | 1 | 0 | [26.718764940159723] | 10731.0 | 1.0 | 1.0 | 87.346694 | -0.085375 | 2.0 | 0.0 |
| 6 | 112.769690 | 119.411429 | 121.029033 | 102.709764 | 112.761271 | 116.961812 | 112.928090 | 113.603236 | 116.378474 | 116.970110 | ... | 2 | 0 | [19.012859718297292, 11.962659024131625] | 12997.0 | 1.0 | 1.0 | 91.323754 | -0.043730 | 2.0 | 1.0 |
| 7 | 123.112274 | 131.304998 | 132.354923 | 121.058213 | 123.104128 | 127.154467 | 123.445751 | 124.079284 | 126.654448 | 127.162771 | ... | 1 | 0 | [32.039676313071055] | 14152.0 | 1.0 | 1.0 | 99.661530 | 0.043576 | 2.0 | 0.0 |
| 8 | 142.439660 | 148.148746 | 149.773579 | 132.386134 | 142.431252 | 147.264814 | 142.722701 | 143.322924 | 146.606358 | 147.273311 | ... | 1 | 0 | [37.11488286759698] | 16459.0 | 1.0 | 1.0 | 86.169050 | -0.097706 | 2.0 | 1.0 |
| 9 | 151.868783 | 161.983193 | 163.033029 | 149.807003 | 151.857001 | 156.399116 | 152.348734 | 153.390386 | 155.899177 | 156.407534 | ... | 2 | 0 | [40.686167878929986, 24.67109827312018] | 17502.0 | 1.0 | 1.0 | 108.372694 | 0.134793 | 2.0 | 0.0 |
10 rows × 44 columns
trials[trials["trial_type"] == "right"][:3]
| start_time | stop_time | InterTrial | SetupTrial | InitializeTrial | WithinTrial | WithinContext | WithinCue | WithinDelay | WithinArms | ... | nSalient | nDistract | posSalient | startIdx | mazeID | mainMazeID | distanceTraveled | excessTravel | rewardFactor | reward | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| id | |||||||||||||||||||||
| 1 | 13.482450 | 23.810661 | 24.893987 | 11.115410 | 13.451317 | 21.935497 | 13.801357 | 14.560017 | 20.968785 | 21.944084 | ... | 2 | 0 | [44.09698487410711, 38.86791360641561] | 1366.0 | 1.0 | 1.0 | 94.579079 | -0.009643 | 2.0 | 0.0 |
| 2 | 35.008142 | 41.995798 | 43.054207 | 24.912208 | 34.978534 | 39.545645 | 35.295272 | 36.178588 | 38.787250 | 39.553976 | ... | 1 | 0 | [42.11787931412284] | 3932.0 | 1.0 | 1.0 | 90.914928 | -0.048011 | 2.0 | 0.0 |
| 3 | 53.151482 | 68.236366 | 69.848842 | 43.074311 | 53.130345 | 57.364091 | 53.513897 | 54.022254 | 56.688921 | 57.372551 | ... | 1 | 0 | [31.783853729187047] | 6075.0 | 1.0 | 1.0 | 92.609304 | -0.030269 | 2.0 | 1.0 |
3 rows × 44 columns
Access TwoPhoton Imaging ¶
This section demonstraces how to access the raw Two Photon imaging data.
NWB organizes data into different groups depending on the type of data. Groups can be thought of as folders within the file. Here are some of the groups within an NWBFile and the types of data they are intended to store:
acquisition: raw, acquired data that should never changeprocessing: processed data, typically the results of preprocessing algorithms and could change
Raw TwoPhoton Imaging¶
The raw TwoPhoton imaging data is stored in pynwb.ophys.TwoPhotonSeries objects (for each channel and plane separately) which is added to nwbfile.acquisition. The data can be accessed as nwbfile.acquisition['TwoPhotonSeries'].
The data in TwoPhotonSeries is stored as a three dimensional array: the first dimension is time (frame), the second and third dimensions represent x and y (width by height).
photon_series = nwbfile.acquisition['TwoPhotonSeries']
# Visualize the imaging data.
from matplotlib import pyplot as plt
plt.imshow(photon_series.data[50].T, aspect="auto", cmap="RdYlBu_r")
plt.title("TwoPhotonSeries")
plt.show()
TODO: times
photon_series.rate
29.873720497348387
photon_series.starting_time
1.2678719758987427
Accessing the segmentation data¶
The segmentation output for the Two Photon Imaging data is stored in nwbfile.processing["ophys"].
In NWB, the PlaneSegmentation class stores the detected regions of interest in the TwoPhotonSeries data. The ImageSegmentation can contain multiple PlaneSegmentation tables, so that we can store results of different segmentation algorithms or different segmentation classes.
We can access the plane segmentation for the TwoPhotonSeries data as
nwbfile.processing["ophys"]["ImageSegmentation"]["PlaneSegmentation"].
plane_segmentation = nwbfile.processing["ophys"]["ImageSegmentation"]["PlaneSegmentation"][:]
plane_segmentation[:10]
| image_mask | ROICentroids | Accepted | Rejected | |
|---|---|---|---|---|
| id | ||||
| 0 | [[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,... | [128, 63] | 1 | 0 |
| 1 | [[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,... | [248, 311] | 1 | 0 |
| 2 | [[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,... | [404, 155] | 1 | 0 |
| 3 | [[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,... | [391, 220] | 1 | 0 |
| 4 | [[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,... | [32, 222] | 1 | 0 |
| 5 | [[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,... | [174, 316] | 1 | 0 |
| 6 | [[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,... | [121, 411] | 1 | 0 |
| 7 | [[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,... | [293, 297] | 1 | 0 |
| 8 | [[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,... | [262, 117] | 1 | 0 |
| 9 | [[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0,... | [218, 284] | 1 | 0 |
plt.imshow(photon_series.data[50].T, aspect="auto", cmap="RdYlBu_r")
plt.title("TwoPhotonSeries")
plt.show()
plt.imshow(plane_segmentation.image_mask[1].T, aspect="auto", cmap="RdYlBu_r")
plt.title("Image mask (single ROI)")
plt.show()
The summary images of the segmentation are stored in Images container in NWB.
images = nwbfile.processing["ophys"]["SegmentationImages"]
images
SegmentationImages (Images)
images (2)
correlation
mean
plt.imshow(photon_series.data[50].T, aspect="auto", cmap="RdYlBu_r")
plt.title("TwoPhotonSeries")
plt.show()
plt.imshow(images.images["correlation"].data[:].T, aspect="auto", cmap="RdYlBu_r")
plt.title("Image Correlation")
plt.show()
plt.imshow(images.images["mean"].data[:].T, aspect="auto", cmap="RdYlBu_r")
plt.title("Image mean")
plt.show()
The fluroscence traces are stored in a Fluorescence container, the raw traces can be accessed as nwbfile.processing["ophys"]["Fluorescence"]["RoiResponseSeries"].
nwbfile.processing["ophys"]["Fluorescence"].roi_response_series
{'Deconvolved': Deconvolved pynwb.ophys.RoiResponseSeries at 0x15896491920
Fields:
comments: no comments
conversion: 1.0
data: <HDF5 dataset "data": shape (100, 481), type "<f4">
description: description of deconvolved traces
offset: 0.0
rate: 10.0
resolution: -1.0
rois: rois <class 'hdmf.common.table.DynamicTableRegion'>
starting_time: 0.0
starting_time_unit: seconds
unit: n.a.,
'Neuropil': Neuropil pynwb.ophys.RoiResponseSeries at 0x15896462736
Fields:
comments: no comments
conversion: 1.0
data: <HDF5 dataset "data": shape (100, 481), type "<f4">
description: description of neuropil traces
offset: 0.0
rate: 10.0
resolution: -1.0
rois: rois <class 'hdmf.common.table.DynamicTableRegion'>
starting_time: 0.0
starting_time_unit: seconds
unit: n.a.,
'RoiResponseSeries': RoiResponseSeries pynwb.ophys.RoiResponseSeries at 0x15896491280
Fields:
comments: no comments
conversion: 1.0
data: <HDF5 dataset "data": shape (100, 481), type "<f4">
description: Array of raw fluorescence traces.
offset: 0.0
rate: 10.0
resolution: -1.0
rois: rois <class 'hdmf.common.table.DynamicTableRegion'>
starting_time: 0.0
starting_time_unit: seconds
unit: n.a.}
Visualize raw traces¶
import pandas as pd
import numpy as np
import pandas as pd
import warnings
warnings.simplefilter(action='ignore', category=pd.errors.PerformanceWarning)
data = nwbfile.processing["ophys"]["Fluorescence"]["RoiResponseSeries"].data[:, :10]
df = pd.DataFrame(data)
df["time"] = np.arange(0, data.shape[0])
df.set_index("time", inplace=True)
df.columns.name = 'ROIs'
import plotly.express as px
fig = px.line(df, facet_row="ROIs", facet_row_spacing=0.01)
# hide and lock down axes
fig.update_xaxes(visible=True, fixedrange=False)
fig.update_yaxes(visible=False, fixedrange=False)
# remove facet/subplot labels
fig.update_layout(annotations=[], overwrite=True)
# strip down the rest of the plot
fig.update_layout(
showlegend=True,
plot_bgcolor="white",
margin=dict(t=10, l=10, b=10, r=10)
)
fig.show(config=dict(displayModeBar=True))